Runtime Testability in Dynamic Highly Available Component-based Systems

Runtime testing is emerging as the solution for the integration and assessment of highly dynamic, high availability software systems where traditional development-time integration testing cannot be performed. A prerequisite for runtime testing is the knowledge about to which extent the system can be tested safely while it is operational, i.e., the system’s runtime testability. This paper defines a cost-based measurement for the estimation of runtime testability, based on a model of the system, independent of the test cases and focused exclusively on test-relevant features of the system. This measurement is used to assist system engineers in directing the implementation of remedial measures, by providing an action plan which considers the trade-off between testability and cost. A low-cost heuristic algorithm is introduced, which allows the computation of near-optimal testability optimisation plans in polynomial time, allowing the application of our approach to systems of realistic size. Two testability analysis are performed on two different component based systems, showing how our approach is used to identify runtime testability problems on a system, and to provide an improvement plan to address their runtime-related testability problems.